Imaging devices such as digital cameras and scanners contain one or more image sensors that are manufactured as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) devices. The image sensor is composed of an array of sensing pixels that captures energy from an illuminant, often converting this energy into an electric measure such as an intensity value. In most cases, imaging sensors will have a certain number of pixel locations that are “defective” due to fabrication or manufacturing errors. A defective pixel of a sensor is one which when exposed to an illuminant will produce a different intensity value or response than that of a fully functional pixel when exposed to that same illuminant. In other words, a defective pixel is abnormally sensitive or insensitive to light.
Pixel defects can be split into three categories: stuck-at-high, stuck-at-low, and abnormal-response. A stuck-at-high defective pixel is one that always responds to the lighting condition by producing a high intensity value. For instance, if the pixel intensity ranges from 0 (low) to 255 (high), a stuck-at-high pixel may always respond to lighting with a value of 248, for example, even if actual measured intensity for that location of the scene would be 31, 50, 100, etc., if captured by a functional pixel. A stuck-at-low defective pixel is one that always responds to the lighting condition by producing a low intensity value. In the same system, a stuck-at-low defective pixel may respond, for example, with a value of 5 even though a functional pixel would show the intensity value to be 200, 100, 78, etc. A pixel with an abnormal response defect has no absolute, but rather a relative variance from a functional pixel. A pixel with an abnormal response defect will inaccurately respond by a particular percentage, such that, where a functional pixel will read a value X, the abnormal response defective pixel will instead respond with a value of 0.25*X, for example. The abnormal response is thus proportionally higher or lower relative to the intensity being captured. Pixels exhibiting any of these types of defects should, desirably, be corrected or compensated for.
Conventionally, the detection of defective pixels is performed in a controlled environment, such as during sensor fabrication, or during camera initialization while the lens cap is still closed. The identified locations are recorded and then transferred to some non-volatile memory on the device in which the sensor is used, such as on a digital camera. The extra memory requirement adds cost and manufacturing complexity to the whole camera system design.
One method of detecting defective pixels during camera operation applies a statistical analysis technique. In the statistical analysis technique, a defective pixel is detected by a binary hypothesis testing scheme, in which, if the likelihood of P1 hypothesis (pixel belongs to “defective” class) is larger than that of P0 (pixel belongs to “normal” class), the pixel is categorized as “defective.” This method does not require memory space to record defective pixel locations, but some offline computation is still required. Additionally, multiple “training” images need to be used to obtain the accumulated likelihood ratio. The selection of the training images needs special care in order to avoid estimation bias.
A more arbitrary way of correcting image defects, which has also been utilized to remove defective pixels during camera operation, is to not detect the defective pixels, but filter out the unknown defects as noise by applying an image-by-image noise removal technique to the entire sensor output (image). The challenge of defective pixel filtering comes from the fact that the distribution of the defective pixels is usually random and the defect locations are usually isolated. Using some form of a linear low pass filter usually does not do well against this type of noise due to its impulse response nature. The result is that a large number of good pixels, or image details, get filtered out along with the bad pixels. The brute-force median or the ranked filter, even though a good solution for salt-and-pepper noise, can unfortunately also filter out image details along with the defects if the filter support is limited.